There is a job management technology for running a batch process that includes a plurality of processes to be run according to an sequential order defined in advance by a user periodically or irregularly in a repeated manner. The batch process is referred to as a “job”. Individual processes which constitute the job are referred to as a “job step”.
In a certain job management technology (for example, see Patent Literature 1), the user is allowed to set an interval between runs for each job step in the job in order to skip job steps which are not necessary to be run every time. In other words, when running a job, running only job steps which have not been run for periods equal to or more than intervals defined by the user since previous times of run is possible.
It is assumed that there is a job including a job step for loading data into a table in a database, and a job step for taking a backup of the table. If the backup is taken every time in this job, running time of each job is increased. With the above-described technology, however, the backup may be set to be taken only once a day.
Two objects as describe below are conceivable for the job management technology which enables setting of intervals between runs.
A first object is, if one of two job steps specified by a user ends abnormally, to maintain the job step in a state of abnormal end so as to allow a retry run at a given timing while establishing a dependency relation therebetween. The dependency relation includes a first dependency relation in which whenever a preceding job step is run in a certain job, a following job step is also run. Alternatively, there is a second dependency relation in which whenever a following job step is run in a certain job, running of a preceding job step has already been completed. If the first object is achieved, safety of the system to which the job management technology is applied is ensured.
For example, it is assumed that there is a job including a job step for extracting data from a table in a database and overwrite the data in the temporary file and a job step for loading the extracted data in another table. In this job, if the job step for data extraction has been run but the subsequent job step for data loading is skipped, a problem of data loss may occur such that the extracted data remaining in the temporary file without being loaded in the another table is erased when a job step for data extraction in a subsequently started job is run. Alternatively, if an abnormal end occurs in the job step for data loading and the job step for data extraction in a subsequently started job is run, a problem may arise such that data failed to be loaded is erased. Therefore, it is necessary not to run the job step for data extraction until a retry run of the job step for data loading is ended normally, and to maintain the state of abnormal end of the job step for data loading while establishing the first dependency relation.
As another example, it is assumed that there is a system having two tables; an accumulation table in which data collected regularly is loaded and a reference table which stores data extracted from the accumulation table and processed. In this system, a job in which five job steps are run in sequence; a job step for loading data in the accumulation table, a job step for running backup of the accumulation table, a job step for extracting data from the accumulation table and overwriting the data in the temporary file, a job step for loading the extracted data into the reference table, and a job step for taking a backup of the reference table is run repeatedly. In this job, consistency between the backup of the accumulation table and the backup of the reference table is maintained by establishing the second dependency relation in which whenever backup of the reference table is run, backup of the accumulation table has already been completed. Therefore, at the time of data restoration, time required for restoring is shortened by restoring the accumulation table and the reference table in parallel. In addition, safety of the system is also enhanced.
As a technology relating to the first object, Patent Literature 1 discloses a method of expressing a dependency relation between job steps only by setting intervals between runs. Patent Literature 2 discloses a method of making a user set a dependency relation between job steps explicitly in addition to the setting of the intervals between runs.
A second object is that if an abnormal end occurs in a certain job step, repeated runs of some job steps before the abnormally ended job step is continued. If this object is achieved, availability of the system to which the job management technology is applied is improved.
As a technology relating to the second object, Patent Literature 3 discloses a method of specifying an abnormally ended job step and executing a retry run.